Exact and inexact subsampled Newton methods for optimization
نویسندگان
چکیده
منابع مشابه
Exact and Inexact Subsampled Newton Methods for Optimization
The paper studies the solution of stochastic optimization problems in which approximations to the gradient and Hessian are obtained through subsampling. We first consider Newton-like methods that employ these approximations and discuss how to coordinate the accuracy in the gradient and Hessian to yield a superlinear rate of convergence in expectation. The second part of the paper analyzes an in...
متن کاملSubsampled Hessian Newton Methods for Supervised Learning
Newton methods can be applied in many supervised learning approaches. However, for large-scale data, the use of the whole Hessian matrix can be time-consuming. Recently, subsampled Newton methods have been proposed to reduce the computational time by using only a subset of data for calculating an approximation of the Hessian matrix. Unfortunately, we find that in some situations, the running sp...
متن کاملInexact Newton Dogleg Methods
The dogleg method is a classical trust-region technique for globalizing Newton’s method. While it is widely used in optimization, including large-scale optimization via truncatedNewton approaches, its implementation in general inexact Newton methods for systems of nonlinear equations can be problematic. In this paper, we first outline a very general dogleg method suitable for the general inexac...
متن کاملAn Investigation of Newton-Sketch and Subsampled Newton Methods
The concepts of sketching and subsampling have recently received much attention by the optimization and statistics communities. In this paper, we study NewtonSketch and Subsampled Newton (SSN) methods for the finite-sum optimization problem. We consider practical versions of the two methods in which the Newton equations are solved approximately using the conjugate gradient (CG) method or a stoc...
متن کاملInexact Newton methods for inverse eigenvalue problems
In this paper, we survey some of the latest development in using inexact Newton-like methods for solving inverse eigenvalue problems. These methods require the solutions of nonsymmetric and large linear systems. One can solve the approximate Jacobian equation by iterative methods. However, iterative methods usually oversolve the problem in the sense that they require far more (inner) iterations...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: IMA Journal of Numerical Analysis
سال: 2018
ISSN: 0272-4979,1464-3642
DOI: 10.1093/imanum/dry009